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(57) Abstract 

A system and method for selective recording of 
information uses sequential partem matching of statistical 
vectors (14) which characterize Incoming time-based 
information to identify previously marked information 
within an incoming information stream. The system and 
method generate a signal to control a recording device ( 1 8) 
to either elide the previously marked information from 
the recording, or to begin recording once the previously 
marked information is identified, depending upon the 
current mode of operation. The system and method may 
be utilized to omit recording of unwanted information, 
such as commercials in a television broadcast, or to 
automatically locate desired information on any of a 
number of available information channels without a priori 
knowledge of the occurrence time or channel of the 
information. 
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SYSTEM AMD METHOD FOR 
SELECTIVE RECORDING OF INFORMATION 

Technical Field 

The present invention relates to a system and 
5 method for selective recording of various types of 
information. 

The explosion of information now available 
relating to a wide variety of topics ranging from 

10 business to entertainment has driven the development of 
technology capable of providing this information to 
various selected groups and individuals. Once the 
information is conveyed from a source to a destination, 
the information is often categorized and stored either 

15 by individuals or by service organizations for 
subsequent retrieval of selected information. 

Regardless of the particular form of received 
information, or the character of the storage media, it 
is often desirable to identify information of interest 

20 and extract that information from the incoming data 
stream for subsequent storage. This procedure optimizes 
the use of the storage media while also increasing the 
efficiency of the user in assimilating the information. 
Thus, the user interested in ascertaining knowledge 

25 relative to a particular area is exposed to the most 
relevant information from a wide variety of sources 
without being inundated with extraneous facts, figures, 
and advertisements. Similarly, a user may select a 
particular audio or video program for recording from 

30 continuously broadcast information so that valuable time 
is not wasted viewing or listening to unwanted 
information. 
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Many people no longer "tune-in" to a single 
broadcast from beginning to end. Instead, people often 
interleave their viewing or listening with "channel 
surfing" and "ad zapping" or "filtering" of information. 
Filtering is commonly called "ad zapping" but is used 
herein to refer to elimination or avoidance of 
advertisements or other repetitive information, such as 
repeated news footage or the like. Surfing, as used 
herein, refers to searching over time and/or available 
information channels for "interesting" information. 



A number of prior art systems have recognized 
the problem of eliminating extraneous information prior 
to recording for subsequent retrieval and review. Many 
of these systems are limited to use with video 
information (often in combination with audio 
information), such as a television broadcast, and are 
not easily adaptable to other forms of information. 

Many of the systems designed for use with 
audio/video (AV) information rely on prior marking or 
coding of the information for subsequent identification 
by a recording device. For example, a television 
broadcast may contain information relative to the 
program characteristics and content such as the starting 
and ending times, the type or content of the program, a 
program guide summary, a classification, a rating, or 
the like. This additional information may be included 
when the signal is originally broadcast, but is 
typically added by an annotation service. The signal is 
received and decoded and may be used to control any of 
a number of devices. For example, closed captioning 
information may be decoded to provide a textual 
transcript of the audio information, or the decoding 
receiver may limit what type of programs are viewed. 
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The signal may also be utilized to automatically record 
particular programs on preselected channels via control 
of a video cassette recorder (VCR) and VCR recording 
tape cassette. Annotation services have been relied 
5 upon to facilitate prior art systems which provide 
surfing functions. 

Reliance upon an annotation service requires 
the viewer to subscribe to the service and forces the 
viewer to explicitly select a menu item which ultimately 

10 corresponds to an approximate time/channel slot. This 
is a continuing expense for the viewer and may not 
support all of the channels which the viewer receives. 
In addition, the interesting material, such as a sports 
broadcast during the nightly news, may not be considered 

15 a "programming unit" by the annotation service so it 
would not be given an identifier which could be used by 
such systems. Furthermore, the interesting information 
may be broadcast on other channels or time slots and 
would not be identified. 

20 Other prior art systems and methods which are 

designed primarily to eliminate recording of commercials 
(zapping) during a program broadcast rely on a change in 
amplitude of the video and/or audio signal components to 
control the recording of the information. The recording 

25 device monitors the incoming signal while recording and 
detects commercials by the characteristic video fade 
and/or audio fade between a program segment and a 
commercial. When a commercial is detected, the 
recording is paused so that the unwanted information is 

3 0 not recorded. Some prior art systems pause for a 
predetermined period of time (i.e. 15 or 30 seconds) 
after detecting a fade (or eliminate material between 
two fades separated by a standard advertisement 
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interval) to avoid recording the advertisement. These 
systems rely on non- intrinsic characteristics of the 
broadcast signal which may lead to incorrect blanks in 
the recording of the desired program material. For 
5 example, an artistic fade in the middle of a program 
broadcast may be mistaken for a fade prior to an 
advertisement resulting in an undesirable 30 second 
blank in the recorded material. In addition, these 
approaches are limited to the elimination of 

10 advertisements and would be difficult to adapt to 
elimination of other "uninteresting" information from 
the incoming information stream, such as repetitive news 
footage. Furthermore, as such systems become more 
prevalent, broadcasters may reduce or eliminate the 

15 fades altogether to defeat operation of these systems so 
that advertisements reach their intended audience. 

Another approach found in prior art systems 
designed to provide automatic recording of a particular 
television program of interest, requires the user to 

20 enter an index code into a VCR which is decoded to 
determine the broadcast channel, program starting time, 
and program duration. Since these systems do not 
monitor the actual broadcast signal being received, once 
recording begins, it continues for the duration 

25 specified regardless of the particular content of the 
signal. These systems are subject to recording errors 
resulting from minor modifications to the programming 
schedule of the television stations.. Such minor 
schedule changes often occur to accommodate important 

30 news broadcasts or the like. 

One alternative approach used to eliminate 
advertisements includes adding control information to 
the recording so that the advertisements are skipped 
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when the recording is replayed. This avoids lost 
information due to an erroneous indication of an 
advertisement but requires the user to wait while the 
recording is fast -forwarded through the advertisement. 
5 Since this approach also relies on audio/video fades, it 
could easily be defeated by broadcasters when the 
economic incentive becomes great enough. Another 
approach offered by at least one television manufacturer 
provides a "real-time 11 zapping function which allows the 
10 viewer to scan other channels for 30 seconds before 
automatically returning to the original channel, so as 
to avoid a standard length advertisement . 

Prior art AV editing systems have utilized the 
highly constrained structure of various news programs to 

15 segment a broadcast based on recognizing predetermined 
logos and faces of newscasters which are broadcast 
repeatedly. After receiving and recording a newscast or 
other structured program, these systems attempt to 
identify the various segments by separating the 

2 0 individual frames into regions and searching for the 
previously identified logos or faces. The segments are 
then classified and archived for use in future programs. 

Disclosure of Invention 

It is thus an object of the present invention 
25 to provide a system and method for selectively recording 
information based on intrinsic characteristics of the 
received information signal . 

It is a further object of the present 
invention to provide a system and method which avoids 
30 recording information which a user has previously 
identified as being uninteresting. 
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Another object of the present invention is to 
provxde a system and method for automatically locating 
and recording information of interest broadcast on any 
one of a number of available broadcast channels. 

Yet another object of the present invention is 
to provide a system and method which automatically 
avoids recording repetitive information. 

Still another object of the present invention 
is to provide a system and method which utilize 
sequential pattern matching to identify previously 
marked information and to control a recording device 
based on the outcome of the pattern matching. 

A still further object of the present 
invention is to provide a data structure particularly 
suited for use in a system and method for selective 
recording of information which facilitates efficient 
unique, and reliable pattern matching of information. ' 

Another object of the present invention is to 
provide a system and method for selective recording of 
information which utilizes sequential pattern matching 
of information based on a summary statistical 
characterization of the information. 



In carrying out the above objects and other 
objects and features of the present invention a system 
is provided which includes a persistent memory for 
memorized information segments (marked by user or by 
repetition) and control logic for computing and matching 
statistical vectors based on the received information. 
The control logic implements sequential pattern matching 
using the statistical vectors to identify memorized 
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information within the incoming information stream. The 
control logic is linked to a recording device to 
selectively record information based on the result of 
the pattern matching. 

5 A data structure is also provided for use with 

the present invention and with other similar systems 
designed to uniquely and reliably identify information 
of interest (or of no interest) for recording (or 
omission from a recording) using sequential pattern 

10 matching. The data structure is organized around 
statistical vectors, each characterizing an information 
"frame." An information frame is a short cohesive 
amount of data. For example, for video, this could be 
a single image field. The data structure includes an 

15 indexable table for each element of the statistical 
vector. Each table provides a list of the information 
frame "identifiers" associated with a given value or 
range of values of that element of the statistical 
vector. Each information frame identifier includes a 

20 statistical vector representing an information frame, a 
pointer to the previously stored frame, and a pointer to 
the subsequently stored frame. The sequence of 
information frames which are joined by 
previous/subsequent frame pointers will be referred to 

25 as an information segment. Depending on the mode of 
operation, the data structure may also include the time 
length to record following the memorized information 
segment or the last date/occurrence count of the 
memorized information segment. 

30 A method is also provided for use with the 

system disclosed and other similar systems. The method 
includes controlling a recording device based on 
sequential pattern matching of information. The method 



WO 96/27983 



PCT/US96/02579 



-8- 

includes marking information of interest, computing a 
sequence of statistical vectors representing the marked 
information, and storing the statistical vectors and 
sequence information to allow future pattern matching. 
5 The method also includes identifying memorized 
information segments within an incoming information 
stream utilizing sequential pattern matching against the 
memorized sequence of statistical vectors, and 
controlling a recording device based on the result of 
10 this pattern matching. 

There are a number of advantages accruing to 
the present invention. For example, the system and 
method of the present invention use summary statistics, 
which capture intrinsic signal information, to detect 

15 repetitive material or information of interest, rather 
than using unintentional signal content, such as audio 
and/or video fades or annotations. As a result, the 
present invention does not require an annotation service 
to provide broadly distributed annotations of the 

2 0 broadcast information. Nor does it rely on 

unintentional, and avoidable, artifacts of composited 
media. 



Another advantage of the present invention 
over those systems and methods found in the prior art is 
that the statistical characterization of the information 
utilized by the present invention will reduce or 
eliminate the possibility of false detections resulting 
in incorrect blanking in the recording of desired 
information. Since the present invention does not rely 
on amplitude changes of the received information, its 
function will not be impaired by incorrect blanking 
caused by unanticipated changes in the. received signal 
during the course of a program broadcast. 
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Also unlike prior art systems and methods, the 
present invention is capable of recording information of 
interest from any of a large number of received channels 
without knowing a priori on which channel or at what time 
5 the information of interest will be broadcast . Many 
prior art systems require the user to enter the 
broadcast time and channel , or a code representing that 
information, which is translated by the recording 
device. In contrast, the present invention does not 
10 require the user to locate such broadcast information, 
but only to provide a representative sample of the 
stable (repeated from installment to installment) 
introductory information and the length of time to 
record once the introductory information is located. 

15 Another advantage of the present invention is 

its applicability to a variety of media sources. The 
present invention is not limited to audio and/or video 
information, or broadcast information in general, but 
may be utilized for sequential pattern matching of 

20 virtually any time-based information signal. For 
example, the present invention may be used in sequential 
pattern matching of seismographic information, pressure 
information, or object profile information (such as used 
in radar and the like) to name a few. 

25 The above objects and advantages in addition 

to other unenumerated objects, features, and advantages 
of the present invention will be readily appreciated by 
one of ordinary skill in the art from the following 
detailed description of the best mode for carrying out 

30 the invention when taken in connection with the 
accompanying drawings . 
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Brief DesgHprjon of t he Drawing 

FIGURE 1 is a flow chart illustrating the 
basic operation of a system and method according to the 
present invention; 

FIGURES 2a and 2b depict data structures for 
use with a selective information recording system and 
method according to the present invention; 

FIGURES 3a and 3b are flow charts illustrating 
information selection for user-marked surfing and 
zapping operation modes according to the present 
invention; 



FIGURE 4 is a flow chart illustrating 
operation of a user-marked zapping mode according to the 
present invent ion ; 

FIGURE 5 is a flow chart illustrating 
operation of a user-marked surfing mode according to the 
present invention; 

FIGURE 6 is a flow chart illustrating 
operation of an automatically-marked zapping mode 
according to the present invention; 

FIGURE 7 is a flow chart illustrating 
information selection for assisted-marking surfing and 
zapping modes according to the present invention; 

FIGURE 8 is a flow chart illustrating 
operation of an assisted-marking zapping mode according 
to the present invention; 
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FIGURE 9 is a flow chart illustrating 
operation of an assisted-marking surfing mode according 
to the present invention; and 



FIGURE 10 is a block diagram representing a 
system for selective recording of information according 
to the present invention. 

Pe$t MQde(s) For Carrying Out The Invention 



It should be recognized that the flowcharts 
presented in Figures 1 through 9 in describing the 

10 present invention depict sequential processing of the 
method steps although any of a number of processing 
strategies could be utilized without departing from the 
spirit or scope of the present invention. For example, 
if the control logic is implemented in hardware, many of 

15 the method steps may be performed simultaneously or 
nearly simultaneously. Similarly, an interrupt driven 
processing strategy could also be utilized to achieve 
the objects and advantages of the present invention. 
One of ordinary skill in the art will also recognize 

20 that the concepts of the present invention may obviously 
be extended to a corresponding parallel implementation 
without departing from the spirit or the scope of the 
present invention. Similarly, a combined 

sequential/parallel implementation utilizing hardware 

25 and/or software to accomplish one or more of the objects 
and advantages of the present invention is within the 
contemplation of the present invention. 

It should also be recognized that, while the 
flowcharts presented in Figures 1 through 9 depict each 
30 of the various surfing and zapping modes operating 
separately and in isolation, the combination of any or 
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all of these methods could be used without departing 
from the spirit or scope of the' present invention. For 
example, assisted-marking zapping and assisted-marking 
surfing may be combined within a single system or 
5 method, by simply marking the individual information 
segments as "for surfing" or "for zapping". 

Referring now to Figure l, a flow chart 
illustrating operation of a system and method for 
selective recording of information according to the 
10 present invention is shown. A user- input request is 
detected at block 8. A current operating mode is then 
selected by the user as represented by block 10. 

The present invention includes two main 
operating modes which share the same computational 

15 engine. The surfing function is used to locate 
previously marked and memorized information segments 
which activate a recording device to record for a 
designated period following the sequence. These 
information segments may be marked by user-marking or by 

20 assisted-marking. The zapping function is used to avoid 
recording previously encountered information segments. 
These information segments may be marked by the user, 
marked through assisted marking, or automatically 
determined by the system. The user-marking modes rely 

25 on user input to determine both the beginning and end of 
the information segments. The assisted-marking modes 
rely on a small amount of user input and assist the user 
in identifying the beginning and end of the information 
segment. The automatic-marking mode (for zapping only) 

30 characterizes and stores all information received while 
continuously comparing incoming information with the 
previously memorized (stored) information to identify 
repetitive sequences. Repetitive information is 
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10 



reinforced while all other information is eventually 
deleted from memory through a "selective forgetting" 
operation as described in detail below. 

The choice between user-marking, assisted- 
marking and (for zapping only) automatic -marking may be 
a design decision where, in any single device, only one 
of the two classes is used for surfing and only one of 
the three classes is used for zapping. However, as is 
obvious from their descriptions, more than one class or 
marking could be supported for either zapping or 
surfing, if that extra flexibility is desired. 



Returning now to Figure l, the user inputs the 
program length at block 11 if the surfing mode is 
selected at block 10. Block 12 represents the learning 
15 (memorizing) of information which includes 
characterizing the information utilizing summary 
statistics and storing the information in an indexable 
table. The information which is learned may be marked 
by the user or automatically marked by the system 

20 depending upon the particular mode selected in step 10. 
Information marked by a user for the surfing mode should 
include stable (repeated from installment to 
installment) identification information, such as opening 
credits or a title screen in the case of video 

25 information. For zapping modes (user-marked, assisted- 
marking, and automatically-marked), marked information 
may include a particular program, advertisement, or 
commercial which the user does not want recorded. of 
course, the present invention is not limited to any 

30 particular type or content of information and is 
applicable to any time-based information signal. 
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While in the automatically-marked zapping 
mode, block 13 determines if the current incoming frame 
is an unmatched frame which should be memorized by block 
12. Sequential pattern matching of received information 
5 to memorized information is then performed at block 14 
of Figure 1. This step is performed autonomously by the 
system without user intervention. Incoming information 
is characterized using summary statistics and then 
pattern matched to previously memorized information. 

10 Block 16 determines if a successful match has been found 
in the incoming information and directs subsequent 
control appropriately. If a match is not found, the 
system continues attempting to locate the marked 
information by repeatedly characterizing the received 

15 information and matching it to the memorized information 
until interrupted by the user. 

When a successful match is indicated at block 
16, a recording device is appropriately controlled as 
represented by block 18. Depending on the mode of 

20 operation which was selected at block 10 when the 
matched information segment was memorized, a successful 
match of the marked information serves a different 
purpose. In the zapping modes, the incoming information 
which matches the memorized information is not recorded. 

25 However, in the surfing modes, the matched incoming 
information initiates the recording process which then 
continues for a designated time period thereafter. 

In modes of operation other than the 
automatically-marked zapping mode, unmatched information 
30 is quickly discarded or "forgotten" and is not 
memorized. In the automatically-marked zapping mode, 
however, all incoming information is memorized for 
comparison to future incoming information. Repetitive 
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information (that which is successfully matched) is then 
automatically zapped and is not recorded. A successful 
match also reinforces the previously memorized 
information. Eventually, unmatched {and therefore 
5 unreinforced) information which was previously memorized 
will be discarded through a "selective forgetting" 
process as required by available memory space. 

A graphical illustration of data structures 
particularly suited for use with the selective 

10 information recording system according to the present 
invention is shown in Figures 2a and 2b. The operations 
of learning (memorizing) time-based information and 
subsequent labeling (matching or indexing) of that 
information are facilitated by these data structures 

15 which employ statistical vectors to characterize the 
information. These vectors are utilized to uniquely and 
reliably identify a segment (or sequence of frames) of 
received information. Uniqueness is required so that 
distinct segments of media are not mistaken for one 

20 another. Reliability is required so that a single 
segment of media which is received twice will match 
itself. The necessary uniqueness may be provided in 
part by careful selection of the statistics utilized to 
characterize the information segment. The sequencing 

25 (order and adjacency) of memorized information also 
provides uniqueness through time that should be 
reproduced when the same material is subsequently 
received. Thus, using a sequence of consecutive frames 
to determine a match improves the confidence of the 

3 0 match so as to reduce the possibility of an erroneous 
matching indication. 

As indicated earlier, a frame of media as 
referred to herein means a short time interval of the 
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media which remains unchanged by its surrounding 
material. For example, a standard frame in video is a 
1/3 Oth- second image along with the coincident 1/3 0th 
second of audio. A "frame" of sound is more difficult 
to define since there are no natural "synchronization" 
marks as are present with conventional video signals. 
A reasonable audio frame may be defined by a 20- 
millisecond section with the assumption that the 
beginning and end frames (which may be contaminated by 
other material) can simply be ignored. For digital 
information, a frame of information may be determined by 
the particular communication protocol used in 
transmission and reception. The operation of the present 
invention does not require any particular, well-defined 
frame length. However, the statistics chosen to 
characterize the information should be particularly 
robust to different framing offsets for audio-only 
sources . 



As illustrated in Figure 2a, a first data 
20 structure, indicated generally by reference numeral 30, 
includes an indexable t table 32 for each element 34 of 
the statistical vector 36. In a preferred embodiment, 
indexable table 30 is implemented using a balanced 
binary search tree as depicted in Figure 2a. However, 
25 one of ordinary skill in the art should appreciate that 
a number of alternative searching implementations could 
be utilized to effect the system and method of the 
present invention, such as arrays, heaps, hash tables, 
and the like. In the preferred embodiment illustrated 
30 in Figure 2a, each balanced binary tree 32 includes a 
plurality of nodes 38 each being characterized by a 
memorized interval, such as interval 40. An interval 
which includes the end-point value is indicated by a 
bracket, " [", while an interval which does not include 
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the end-point value is indicated by a parenthesis, ")". 
Thus, the interval [4 7) includes values greater than or 
equal to four (4) and less than seven (7) , but not equal 
to seven (7) . As is known in the computer-related arts, 
5 each binary tree may be balanced to increase searching 
efficiency when locating matching statistics of desired 
information. Thus, as each tree grows, the depth of the 
tree is balanced using the process of rotation to put 
the root of the tree at the center of the leaves so that 
10 no branch extends more than one leaf deeper than any 
other branch. A detailed explanation of this strategy 
is provided by Robert Sedgewick, Algorithms, pp. 187-201 
(Addison-Wesley 1983) . 

The original intervals are formed when an 

15 information frame is memorized by adding and subtracting 
an error margin to the computed statistical value for 
each statistical element 34 in statistical vector 36. 
The error margin, and thus the span of the original 
interval, depends upon the level of noise expected in 

20 the received signal (which may depend on the type of 
signal, i.e. analog or digital). An appropriately 
selected error margin increases the reliability of the 
pattern matching process . An analog signal may require 
a wider interval for satisfactory operation than the 

25 interval necessary for the same quality of operation 
when receiving a digital signal. If the error margin is 
too small for the quality or type of the received 
information signal, information which should match 
previously identified material will not be detected. If 

30 the error margin is significantly larger than necessary, 
the system will identify information segments which have 
similar statistical representations but which are not 
intended to be matched. This results in an inefficient 
matching process and may result in erroneous matches. 
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The particular statistics utilized to 
characterize the received information should be chosen 
based upon the type of information and the type of 
signal received. In a preferred embodiment of the 
5 present invention, the received signal is a compressed 
audio and video signal which conforms to the Joint 
Photographic Experts Group (JPEG) compression strategy, 
which is well known in the art. The set of statistics 
utilized in this case may be as simple and readily 

10 available as the spatially-aliased, bandpass thumbnail 
sketch of the received information frame. For example, 
each statistical element 34 may be obtained from the 
low- frequency coefficients in two-space (not including 
the zero frequency or DC coefficient) of an eight by 

15 eight block (of pixels) discrete cosine transform (DCT) 
used in JPEG or MPEG (Motion Picture Experts Group) 
compression along with the compressed audio signal 
corresponding to the video frame period. As is known in 
the art, the DCT is only one of the many available fast 

2 0 unitary transforms which may be used. The block DCT 

transform is particularly suited to this application 
since it compresses most of the entropy of the 
information into a small number of coefficients which 
may then be utilized as a unique summary 
25 characterization of the information segment. Some 
examples of other transforms which may be used include 
the wavelet, the block-Hadamard, and the block-Karhunen- 
Loeve transforms. 

Alternatively, simpler statistics may be 

3 0 utilized depending on the information signal, such as 

row and column averages. Of course, more than one type 
of statistic could be used to determine one or more of 
the statistical elements 34 in statistical vector 36 so 
as to uniquely characterize the received information. 
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The sensitivity of the selected statistic to noise in 
the received signal may also influence selection of the 
interval size. While the error intervals, such as 
interval 40, are preferably centered about the value of 
5 a corresponding statistical element 34, other offsets 
may be utilized without departing from the spirit or 
scope of the present invention. Similarly, the 
intervals may include only a single value. 

With continuing reference to Figure 2a, as 
10 intervals 4 0 are stored, they are subdivided into 
multiple, non- overlapping intervals as each indexable 
table (binary tree in a preferred embodiment) grows in 
such a way that the final stored intervals associated 
with the outermost nodes are uniform as well as non- 
15 overlapping. In a preferred embodiment, each node 38 of 
a binary tree 32 points to a list of one or more frame 
identifier names (or titles) 42 associated with the 
value or values contained within the interval 
corresponding to that node. 

20 As illustrated in Figure 2b, a frame 

identifier, such as frame identifier 50, is another type 
of data structure utilized in the system and method of 
the present invention. A frame identifier includes a 
statistical vector having statistical element values 

25 corresponding to a particular frame of received 
information and pointers to the frame identifiers which 
correspond to the adjacent frames (precedent 52 and 
subsequent 54) in the originally received time sequence, 
if those frames have been memorized. If the frame is 

30 the first or last in a particular sequence, the pointers 
contain a zero or null indication for the previous 52 or 
subsequent 54 frame identifier, respectively. The frame 
identifiers also contain a value 56 representing the 
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minimum number of sequential frames which must be 
matched to be considered a successful match, the time 
length of information 58 to follow a particular sequence 
of frames (and therefore the length of time to record 
5 interesting information), and the last occurrence 60 and 
frequency 62 of a frame matching that particular frame 
identifier. 

Some fields of each frame identifier are not 
used in all the available operating modes. Thus, a 

10 system of diminished functionality could be produced 
using a data structure without those corresponding 
fields. For example, the program length field 58 is 
used only by the surfing functions to determine the 
length of time to record after a successful match has 

15 been indicated. The last occurrence 60 and frequency 
fields 62 are used only by the automatically-marked 
zapping mode of operation. Thus, an appropriate data 
structure is created including only the necessary fields 
as indicated by the current operating mode. 

20 Referring now to Figure 3a, a flow chart 

illustrating information selection for user-marked 
surfing and zapping operation modes according to the 
present invention is shown. At step 80, the user 
initiates the process by marking the start and end of an 

25 information segment which is to be memorized and later 
sought to begin a recording session or to be omitted 
from a recording. For example, the user may mark the 
beginning credits of a favorite program or the user may 
mark a disliked commercial advertisement. The user 

3 0 indicates the minimum length of a match against this 
desired information at step 82 by selecting the 
percentage of the marked information sequence which must 
be matched for the match to be considered successful. 
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Thus, a match may be considered successful even if the 
very beginning of the sequence is not observed without 
allowing a single frame match to trigger either the 
zapping process or the surfing process- The beginning 
5 of the sequence may not be identified when, for example, 
a user initiates zapping or surfing part way through the 
broadcast of a memorized information segment or when a 
user- input session interferes with the autonomous part 
of the zapping and surfing modes, 

10 As illustrated at step 84 of Figure 3a, if the 

surfing mode has been selected, the user must input the 
program length at step 86. The program length 
represents the period of time to operate an attached 
recording device to record the desired program after a 

15 successful match of the information segment has been 
indicated. At step 88, a statistical vector is computed 
for the current frame of information based on the 
selected set of statistics as discussed with reference 
to Figures 2a and 2b. At step 90, a frame identifier 

20 ( illustrated in Figure 2b) is created and recorded 
within the memorized (stored) tables, as explained in 
greater detail with reference to Figure 3b. If the end 
of a marking sequence is indicated at step 92, the input 
process is complete and the system and method continue 

25 by autonomously searching for the marked information 
segment (and previously marked information segments) 
within the incoming information stream. 

Otherwise, the next frame of information is 
obtained as indicated by step 94 and the process 
30 continues with step 88. Steps 88 through 94 are 
repeated until all the frames of the marked information 
segment are recorded both as a sequence of consecutive 
frame identifiers and as nodes in the indexable table. 
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After the input process has been completed, control 
passes to the appropriate process for the current 
operational mode as indicated by step 96. The various 
operational modes are illustrated and described in 
detail with reference to Figures 4 through 9. 



Figure 3b provides a detailed illustration of 
a process for creating a frame identifier. The process 
begins at step 90a by allocating space for the data 
structure. The statistical vector characterizing the 
associated received information frame is recorded within 
the data structure at step 90b. The minimum length for 
a successful match is stored at step 90c and the 
adjacent frame pointers (such as previous pointer 52 and 
subsequent pointer 54) are initialized at step 90d. 
This may include entering a null value for the first or 
last frame in a particular sequence as discussed with 
reference to Figure 2b. If surfing mode is active as 
indicated by step 90e, the recording time for the 
program of interest is stored within the frame 
identifier at step 90f . Otherwise, control is passed to 
step 90g without entering the program length 
information. 



Preferably, the user marking and memorizing 
processes illustrated in Figures 3a and 3b are completed 
on a system with sufficient resources so that the 
computational throughput is equal to or greater than the 
frame rate. The throughput and latency of the creation 
of the frame identifiers, the computation of the error 
intervals and the creation of the indexable table from 
these intervals do not have to occur at the information 
reception rate, as long as the frame statistics are 
buffered in sequence until these operations are 
completed. 
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The system utilized to implement the user 
marking and memorizing process should have sufficient 
memory to buffer the frame statistics until the 
processing, or memorizing is completed, i.e. until the 
5 appropriate intervals and sub- intervals are created and 
the statistics are properly located within the indexable 
table. A preferred system also includes sufficient 
memory to accommodate a large number of frame 
identifiers since a frame identifier must be retained 

10 for each frame in the marked sequence throughout the 
operational life of the system, or until they are 
deleted by the user. The memory should also preferably 
be sized to accommodate a search tree for each element 
of the vector of frame statistics throughout the 

15 operational life of the system. 

Referring now to Figure 4, a flow chart 
illustrating the user-marked zapping function according 
to the present invention is shown. Once information has 
been selected as uninteresting by the processes 

20 illustrated in Figures 3a and 3b, the user-marked 
zapping function enters an independent mode of 
operation. One object of this process is to pattern 
match the incoming information frames (which are being 
recorded by a recording device) against the memorized 

25 sequences. If a match of sufficient length is detected, 
the recording is affected appropriately. This may 
include reversing or rewinding to the point where the 
match was first detected. Alternatively, control 
information may be added to the recording so that the 

30 marked information is skipped during subsequent playback 
or a delay buffer, inserted between the incoming 
information stream and the recording device could be 
emptied, with the recording paused until it refilled. 
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A frame of information is obtained from the 
incoming information channel being recorded at step 100. 
A statistical vector is calculated at step 102 using the 
same statistical calculations which were utilized during 
the user marking and memorizing process of Figures 3a 
and 3b. Step 104 compares the current statistical 
vector with previously encountered statistical vectors. 
The tables of memorized statistics are examined to 
determine if the same or a similar set of statistical 
values were already encountered, preferably utilizing 
balanced binary search trees corresponding to each 
element of the statistical vector. 



In a preferred embodiment, indexing into each 
binary tree is completed by finding all the leaves of 

15 the tree having an interval which contains the value 
computed for the current statistical element. For the 
first element and the first binary tree, the 
identification numbers which have been indexed in this 
manner are placed into a temporary cache. For each of 

20 the remaining elements and binary trees, the 
identification numbers in the temporary cache must be 
found among the indexed leaves, otherwise they are 
deleted from the cache. Step 106 determines when the 
temporary cache becomes empty, which indicates that the 

25 current statistical vector does not match a previously 
marked statistical vector and the indexing process 
terminates. Once the indexing process terminates, a 
data counter, which indicates the location of the 
current frame within the incoming information stream, is 

30 incremented and the program control returns to step 100; 

At step 108 of Figure 4, the starting location 
(i.e. frame number or position) for a memorized 
recording is cached. This will be used in rewinding or 
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reversing the recording to delete the memorized 
information detected within the incoming information 
stream if a sufficient number of sequential frames are 
matched. Step 110 then obtains the next frame of 
5 information and step 112 computes a corresponding 
statistical vector for the subsequent frame in the same 
manner as step 102 . Step 114 compares this statistical 
vector to the statistical vector of the subsequently 
memorized frame identifier as indicated by the 
10 corresponding subsequent pointer (54 of Figure 2b) . 
Since the subsequent frame is indicated by a pointer, 
indexing into the memorized tables is not required. 

As indicated by step 116, if the next 
memorized frame statistics match the statistics computed 

15 in step 112, then step 118 determines if the end of the 
memorized sequence has been reached as indicated by a 
null value in the subsequent frame field 54 of the frame 
identifier. If the end of sequence is not indicated, 
control continues by incrementing the data counter to 

20 select and process the next frame within the incoming 
information stream at step 110. Otherwise, if step 116 
determines the current frame does not match the 
subsequent frame of the memorized sequence, or step 118 
determines that the memorized sequence has ended, 

25 control continues at step 120. 

The length of the matched sequence (i.e. 
number of frames) is compared with the minimum match 
length of the selected memorized sequence at step 12 0 to 
determine if the match is long enough to indicate 
3 0 success. In the user-marked zapping mode, the 
determination as to whether the matched sequences were 
"long enough" depends on both the minimum match length 
indicated by the user in step 82, and on the constraints 
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imposed by the computational latency. For example, if 
the latency of steps 102 and 104 is longer than a frame 
period and the latency in the loop of steps 110 through 
118 is less than a frame period, the "long enough" 
constraint includes the need to make up for the excess 
latency of steps 102 and 104 and to thereby "catch up" 
to the current frame. 

If the match is long enough as determined at 
step 120, step 122 controls the recording device to zap 
the marked information from the recording. This may be 
completed by returning to the starting location which 
was stored at step 108, such that the intervening 
information frames will be deleted or subsequently 
rerecorded and therefore lost, or by recording a control 
code to skip the marked information upon playback or by 
emptying a delay buffer, inserted between the incoming 
information stream and the recording device and pausing 
the recording until it refilled. If the result of step 
12 0 is negative, the recording device continues 
uninterrupted recording and the process is repeated with 
step 100 to search for a matching sequence. 

The computational capabilities of a system 
implementing the user-marked zapping function should 
provide a latency for computation of the frame 
statistics and the comparisons which is less than a 
single frame period. The latency of indexing into the 
memorized tables can be greater than a single frame 
period as long as its computational throughput is 
greater than the frame rate, and the latency delay is 
compensated for by the simple frame statistics 
computation/comparison loop (i.e. steps 110 through 118) 
in less than the time length of the marked sequence. An 
implementational system should also have sufficient 
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memory to buffer the current frame statistics until such 
time as processing can be completed. The system should 
also have a cache for frame identifier numbers to 
provide for indexing across the separate memorized 
5 tables. 

Figure 5 illustrates the user-marked surfing 
function of the system and method for selective 
recording of information according to the present 
invention. The surfing function allows a user to find 

10 and record material which follows a segment which has 
been previously marked. After the information is 
marked, the surfing function continually scans the 
available channels to find the designated material . 
Once that material is located, subsequent information is 

15 recorded for a predetermined period of time as indicated 
by the user. 

Step 140 of Figure 5 selects a current channel 
to search for the marked information. This step may be 
done in a variety of different ways including random 

20 selection, unbiased scheduling, or biased scheduling, 
among others. Random selection could utilize a random 
number generator to select the current channel. 
Unbiased scheduling would follow a predetermined channel 
order such as stepping through available information 

25 channels using channel numbers as an ordering function. 

Biased scheduling would modify unbiased scheduling by 
monitoring preferred channels more often than other 
channels. Preferred channels could be selected by the 
user or they could be selected automatically by the 

30 system based on past success in finding programs 
containing information of interest. 
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With continuing reference to Figure 5, step 
142 obtains the current frame of information from the 
incoming information stream. A statistical vector is 
computed at step 144 to characterize the incoming 
5 information frame and compare it to previously memorized 
frames at step 146. Steps 148 through 158 process the 
incoming information frames in a similar manner as 
previously described steps 106, 110 , 112, 114, 116, and 
118, respectively. Step 160 determines whether a match 
10 is "long enough" to indicate success based only on the 
minimum match length indicated by the user during the 
information identification process illustrated in 
Figures 3a and 3b. 

Step 162 controls the recording device to 
15 begin recording information and continue for the 
predetermined time period specified by the user during 
the information marking process. In one embodiment, the 
search for other memorized information is suspended for 
the period during the recording. In another embodiment, 

2 0 if more than one sequence has been marked for recording 

and a preference is indicated among the marked 
sequences, the system continues to monitor available 
information channels while recording to locate 
information having a higher preference code. If such 
25 information is located, the current recording can be 
terminated so the higher preference material may be 
recorded. 

The computational and memory requirements for 
a preferred system implementing the user-marked surfing 

3 0 function are directly proportional to the computational 

and memory requirements for the user-marked zapping 
function. The proportionality constant varies between 
unity and the number of available information channels, 
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10 



depending on the need to scan multiple channels in a 
single frame period. This requirement, in turn, depends 
upon the length of the marked information and the 
percentage match required to indicate a successful 
match . 

Referring now to Figure 6, a flow chart 
illustrating an automatically-marked zapping function of 
a system and method for selective recording of 
information according to the present invention is shown. 
Automatically-marked zapping has the same goal as user- 
marked zapping: eliding repetitive material from a 
recording. instead of requiring the user to mark the 
material, however, automatically-marked zapping keeps 
track of all the information which has been received in 
15 the recent past. 

Step 170 of Figure 6 obtains the next frame of 
information for step 172 to compute an appropriate 
statistical vector as previously described. step 174 
compares the current frame statistical vector with 
previously memorized statistical vectors as previously 
described. If the current statistical vector does not 
match any of the memorized vectors as determined by step 
176, the control logic proceeds to step 178 which 
creates and memorizes a new frame identifier. For the 
25 automatically-marked zapping mode, the created frame 
identifier data structure includes a time stamp 
indicating the date and time at which the information 
was recorded and a frequency counter which monitors the 
number of times a frame matching that particular 
30 statistical vector has been detected. The frame 
identifier is then created as illustrated in Figure 3b 
by allocating space, initializing the pointers, and 
recording the statistical vector. However, no minimum 
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match length is recorded in this frame identifier. 
Instead, a single, global value is used. Once the frame 
identifier structure has been created, it is stored in 
the memorized tables as previously described with 
reference to the user-marked zapping function. 

Step 180 is performed only periodically to 
selectively forget information so as to reduce the size 
of the memorized tables. Otherwise, the size of the 
memorized tables would grow without bound the longer the 
automatically-marked zapping mode was active. To avoid 
difficulties associated with using all the available 
memory, and increasingly difficult indexing of 
information, the size of the memorized tables is thus 
periodically reduced. 



The memorized tables are reduced in size by 
traversing the memorized tables, examining each frame 
identifier to determine the last time it was received 
and the frequency of recognition. The frame identifiers 
which are too old or too infrequently encountered are 
then "forgotten". The process of "forgetting" a frame 
identifier includes removing all pointers to that 
identifier (in their predecessor and successor fames and 
in the memorization tables) and allowing the memory 
occupied by that frame identifier to be reallocated by 
the system. The control logic then returns to step 170 
and the process continues. 



With continuing reference to Figure 6, if step 
176 indicates a match with a previously memorized frame, 
step 182 caches a starting position or frame so the 
recording device may be reversed or rewound if a 
successful sequence match is indicated as previously 
described. Step 184 updates the occurrence and 
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frequency counters in the appropriate frame identifier 
data structure which matched the current statistical 
vector. 

With continuing reference to Figure 6, step 
5 186 obtains the next frame of information while step 188 
computes the statistical vector and step 190 compares 
the statistical vector to the next memorized frame in 
the sequence. Step 192 determines if the current frame 
matches the next memorized frame, as previously 

10 described with reference to steps 150 through 156 of 
Figure 5. Step 194 updates the occurrence and frequency 
information similar to step 184 if the next frame is 
matched as determined by step 192. Subsequent frames 
are analyzed until the end of sequence is indicated by 

15 step 196. Step 198 determines whether a matched 
sequence is of sufficient length to be considered 
successful as previously described with reference to 
user-marked zapping, noting that the minimum match 
length is now determined using the previously mentioned, 

20 global number. 

Step 202 of Figure 6 then controls the 
recording device to automatically zap the repetitive 
material from the recording. Similar to the user-marked 
zapping mode, zapping may be performed by returning to 
25 the position stored at step 182, or by recording control 
codes which skip over the marked information upon 
playback or by emptying a delay buffer, inserted between 
the incoming information stream and the recording device 
and pausing the recording until it refilled. 

30 The computational requirements of a preferred 

system implementing the automatically-marked zapping 
function are the same as those described for the user- 
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marked zapping function with the additional requirement 
that the throughput of creating and memorizing a new 
frame identifier must be greater than the frame rate. 
The computational requirements of step 180 to 
5 selectively forget frame identifiers will not be 
constrained as long as this step is only occasionally 
necessary. In that case, the reduction can be completed 
when there is no incoming information frames (i.e. the 
recording device is off or the automatically-marked 

10 zapping function is disabled) . If necessary, the 
reduction process can occur in the middle of a zapping 
session, by temporarily suspending the zapping function. 
During that time, repeated video would not be zapped 
from the recording and none of the incoming information 

15 frames would be memorized. 

The memory requirements of a preferred system 
implementing the automatically-marked zapping function 
of the current invention are similar to those described 
to implement the user-marked zapping function. However, 

20 the memorizing process is continuous and newly memorized 
frames are preferably retained for at least the minimum 
expected repetition period before they are forgotten 
during a memorized table reduction. For example, a user 
who wishes to avoid recording advertisements during 

25 video recording may activate the automatically- marked 
zapping function during regularly recorded programs. If 
the program material is about two hours per day and the 
advertisements that are shown during those programs are 
repeated daily, then the automatically-marked zapping 

30 function would have to have enough memory to retain the 
frame identifiers and the indexing tables to accommodate 
just over two hours of video information. This would 
allow the advertisements to be seen more than once 
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before their statistical vectors and frame identifiers 
would be removed due to memory constraints. 



Referring now to Figure 7, a marking and 
memorizing process for assisted-marking zapping and 
5 surfing modes of a system and method for selective 
recording of information according to the present 
invention is shown. This user-input portion is similar 
to the user-input portion of the user-marked zapping and 
surfing functions illustrated and described with 
10 reference to Figures 3a and 3b. Assisted marking thus 
reduces the amount of effort that is required from the 
user to mark an information index without significantly 
increasing the amount of required system memory. 

At step 22 0 of Figure 7, a frame of 

15 information is obtained and step 222 computes a 
statistical vector representative of the information 
within the received frame. A statistical vector is 
calculated for each incoming information frame instead 
of only the frames occurring between a user- specif ied 

20 starting and ending point as illustrated in Figure 3a. 
At step 224, the statistical vector is buffered, 
preferably in a circular buffer, to accommodate a 
predetermined response time of the user (equivalent to 
the "recall" length of the buffer) . This provides a 

25 sequence of a predetermined number of previous frame 
statistical vectors which is available at all times. 
This allows for some delay. A starting point is 
estimated as being the recall length before the user 
mark. An ending point is estimated as being the input 

30 match length after the user mark. The input match 
length is input at step 230. The starting and ending 
points along with intervening frames are then memorized 
as explained in greater detail below. It is not 



WO 96/27983 



PCT/US96W2579 



-34- 

necessary to compensate for the response time with any 
precision since chunking and sequence extension 
(explained below with reference to Figures 8 and 9) 
adjust the boundary points of the memorized segment. 

5 step 226 determines if a user input has been 

received indicating the desire to mark information. If 
no user input is received, a statistical vector is 
computed on the next frame of information and stored in 
the circular buffer by repeating steps 220 through 224. 
10 Once the circular buffer is full, the statistical vector 
of the next following frame replaces the oldest (in 
time) received frame in the buffer, etc. Once user 
input is received, step 228 marks the position within 
the circular buffer of the indicated starting frame. 

15 At step 230, the user inputs the minimum 

number of frames to indicate a successful match. This 
information is used to allow a successful match even if 
the very beginning of the sequence is not observed and 
also to determine the minimum amount of information to 

20 be collected and memorized for later marking. The 
latter use is necessary for this mode since the user 
-does not -indicate a starting and ending point, but 
provides only a single indication to mark the sequence . 
If surfing mode is indicated at step 232, the program 

25 length is entered at step 234 and the control logic 
continues with step 236. Otherwise, step 234 is skipped 
and the process continues with step 236. 

Step 236 moves the circular buffer pointer 
backward (as measured by time) through the circular 
30 buffer a length equal to the recall length. Thus, the 
oldest statistical vectors can be memorized before they 
are overwritten by vectors characterizing subsequently 



WO 96/27983 



PCT/US96/02579 



-35- 

received information frames. This also allows 

information received before the user-marked starting 
frame (saved at step 228) to be inserted into the 
memorized tables by steps 238 and 240. Subsequent 
5 frames continue to have their statistical vectors 
calculated and stored in the circular buffer while steps 
238 memorizes the statistics until step 240 detects the 
end of the sequence. Unlike the user-marked input 
process illustrated in Figures 3a and 3b # the assisted- 

10 marking input process of Figure 7 automatically 
determines the endpoint of the sequence as being the 
minimum sequence length for a successful match 
(determined at step 230) after the user-marked point 
(determined at step 228) . After step 24 0 indicates the 

15 end of sequence has been memorized, control then 
proceeds via step 242 to the appropriate process 
illustrated in Figure 8 or 9 depending upon the current 
operating mode. 

Referring now to Figure 8, a flow chart 
20 illustrating the autonomous portion of an assisted- 
marking zapping function for a system and method 
according to the present invention is shown. The 
assisted-marking zapping function and the assisted- 
marking surfing function (illustrated in Figure 9) are 
25 similar to the user-marked zapping and surfing 
functions, respectively. The difference is that the 
assisted-marking functions provide for automatically 
extending and/or trimming of the memorized sequences 
using the incoming information stream until the 
30 memorized sequences consist of individual, cohesive 
units of information. A cohesive unit describes an 
information segment which is always observed as a unit. 
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Extension of the memorized segments occurs 
when a terminal frame (the first or last frame) of 
memorized information is matched against a similar frame 
in the incoming information. The memorized segment is 
5 extended either forward or backward in time (if the last 
or first frame was matched, respectively) by memorizing 
additional sequential frames from the incoming 
information stream beginning with the frame adjacent to 
the matched terminal frame. Extension of the memorized 
10 segment is stopped when the chunking process (step 290) 
finds its segment boundaries. 

Thus, steps 260 through 266 compute 
statistical vectors based on the incoming information 
stream and compare them to previously marked and 

15 memorized frames as previously described with reference 
to the user-marked zapping function (Figure 4) . If the 
information was not previously marked (i.e. the current 
frame does not match any of the memorized frames) , step 
268 buffers the statistics in a circular buffer. This 

20 computation and circul ar buf f er may be shared between 
steps 222-224 of Figure 7 and steps 262-268 of Figure 8, 
since the same information is being computed and 
buffered in each place. This buffering of statistics 
allows extension of the memorized sequences backward 

25 through time as explained below. However, any gaps 
within the circular buffer (indicating information 
frames on which step 262 was not completed) will force 
a "fill counter" for that buffer to zero to indicate 
that there is a break in the data within that buffer. 

30 Furthermore, gaps of this type force an "extension 
counter" to zero to indicate that sequence extensions 
should not be continued (since the extension requires 
contiguous computations from step 262) . The purpose of 
the extension counter will be discussed below. 
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Step 270 is illustrated with a dashed box 
since it is only performed when a recent successful 
match (one that is long enough) ended at the end of a 
memorized segment. This is indicated by a positive 
5 value for the extension counter which may be set by step 
286 which is explained below. Thus, if the extension 
counter is zero, step 270 simply returns control to step 
260. When the extension counter is positive, step 270 
extends the memorized sequence forward in time by 

10 creating and memorizing frame identifiers using the 
current statistics determined at step 262 . The frame is 
then added to the end of the memorized sequence that was 
last matched using an extension pointer (from step 286 
explained below) to retrieve the last frame in that 

15 sequence. The extension pointer is then updated to 
point to the frame identifier that was just created and 
is now at the end of the matched memorized sequence. 
The extension counter is then decremented. 

With continuing reference to Figure 8, step 
20 272 caches the starting location within the incoming 
information stream of the first marked frame for use, if 
necessary, by step 292. As indicated by a dashed box, 
step 274 is performed only when the start of a new match 
is found at the start of a memorized segment. This step 
25 is aborted before modifying any memorized information if 
a match does not occur at the start of the memorized 
data (i.e. if the first matched frame of the incoming 
information stream corresponds to a memorized frame 
which does not have a null value for the previous frame 
30 pointer) . Furthermore, it is also aborted if the 
matched starting frame is a "boundary frame". The 
meaning of the term "boundary frame" is described under 
step 290, below. Finally, the effects of step 274 are 
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reversed by step 294 if the match is determined to be 
too short by step 288. 

In contrast to step 270, step 274 attempts to 
extend the memorized sequence backward in time. Step 
5 274 attempts this extension only if the matching 
memorized frame occurs at the start of a memorized 
sequence, the matching frame is not a boundary frame, 
and the circular buffer of statistics includes valid 
time -contiguous statistics from the incoming information 

10 stream. When these conditions are met, the memorized 
sequence is extended backward in time by creating and 
memorizing a sequence of frame identifiers from the 
statistics buffered by step 268 and attaching them to 
the start of the previously memorized sequence. The 

15 point of attachment of the added sequence is cached so 
that the effects may be reversed by step 294 if 
necessary. Without this ability, incorrect short - 
sequence matches would add unrelated information onto 
the front of the memorized sequence. Steps 276 through 

20 284 proceed in a manner similar to that illustrated and 
described with reference to steps 110 through 118 of 
Figure 4 . 



If the end of a memorized sequence is detected 
at step 284, then step 286 determines whether it is 

25 appropriate to extend the memorized video forward in 
time (as explained with reference to step 270) . If the 
end of the matched memorized sequence is not a boundary 
frame (as defined below under step 290) , the memorized 
information should be extended forward in time. In this 

30 case, step 286 sets the extension pointer to the 
currently matched, memorized frame identifier, and the 
extension counter to the maximum extension length. This 
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allows step 270 to extend the trailer if the other 

conditions for extension are met as explained above. 

If the matched sequence is long enough as 
determined in step 288, step 290 updates the "chunking" 
5 of the memorized information. This step updates the 
sequencing information between the memorized frames that 
were matched and the memorized frames which were not 
matched so that the sequences contained in the memorized 
tables only cover pieces (chunks) of information which 

10 always occur together. The starting point of the 
matched sequence will form a "segment boundary" provided 
there was incoming information (which failed to match) 
immediately preceding that starting frame. In this 
case, the pointers which cross this segment boundary are 

15 nulled which indicates that the corresponding frame 
identifiers on either side of the segment boundary are 
"boundary frames". Similarly, the ending point of the 
matched sequence is updated. Thus, if memorized 
information follows the matched information, the 

20 pointers across this segment boundary are nulled and the 
frame identifiers on either side of the segment become 
boundary frames as well. Once the memorized sequence is 
divided into chunks (if necessary) , only the chunk that 
contains the user-marked frame is retained. Other 

25 chunks are forgotten by removing all pointers to the 
frames in those chunks and making the memory space 
available to the system. In addition to assuring that 
the memorized tables contain only pieces of information 
which always occur together, this process also stops the 

30 growth outward at the boundary frames during automatic 
leader/trailer extension explained below. 

Step 292 controls the recording device to zap 
the information from the recording as explained with 
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reference to step 202 of Figure 6. If the matched 
sequence is not long enough as determined by step 288, 
step 294 is performed. Step 294 eliminates any frames 
added by step 274 since the matched sequence was too 
5 short for extension. This elimination involves 

forgetting the leader extension by removing all pointers 
to the frames in that extension and making the memory 
space available to the system. 



Referring now to Figure 9, a flow chart 
10 illustrating an assisted-marking surfing function of a 
system and method for selective recording of information 
according to the present invention is shown. This 
function is similar to the assisted-marking zapping 
function illustrated in Figure 8 and the user-marked 
15 surfing function illustrated in Figure 5. Once 
information is marked by the process illustrated in 
Figure 7, a channel is selected at step 300 in an 
attempt to locate the marked information. This channel 
selection process is the same as discussed for step 140 
20 of Figure 5. 



Steps 302 through 310 continue buffering 
statistical vectors of incoming information frames until 
a frame matches a previously marked frame as determined 
by step 308. To avoid confounding of information 

25 segments, step 310 preferably utilizes a different 
buffer for each information channel. Of course, other 
techniques for associating an information frame with the 
information channel which it came from are possible. As 
with step 270 of Figure 8, step 312 of Figure 9 is 

30 performed only when all its conditions are satisfied 
which may occur only after step 326 has properly 
initialized the extension pointer and counter. Steps 
314 through 330 function in a similar manner as that 
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described with reference to steps 274 through 290 of 
Figure 8. Step 332 controls a recording device to begin 
recording the incoming information for the time 
specified by the user. Step 334 performs a similar 
5 function to that described in reference to step 294 of 
Figure 8 in that step 3 34 reverses the effects of step 
314 if necessary. 

Referring now to Figure 10, a block diagram 
illustrating a system for selective recording of 

10 information according to the present invention is shown. 
Preferably, control logic 34 0 is implemented by a 
computer programmed to effect system operation as 
illustrated in Figures 1 through 9. Of course, the 
system and method of the present invention may be 

15 effected with control logic implemented utilizing a 
number of combinations of general purpose and special 
purpose hardware and software, such as application- 
specific integrated circuits (ASICs) , reduced 
instruction set controllers (RISCs) , programmable logic 

20 arrays (PLAs) , discrete components, or the like. 

A persistent (non-volatile) memory 342 
receives and stores incoming information and is in 
communication with control logic 340 . Persistent memory 
342 may include a significant amount of homogeneous or 

25 heterogeneous memory such as DRAM, RAM, EEPROM or the 
like. The preferred size and preferred computational 
capabilities of persistent memory 342 and control logic 
34 0, respectively, have been described with reference to 
their associated functions illustrated in Figures 3 

30 through 9. Input device 344 may be an integrated user 
interface to allow entry of instructions and selection 
of user-marked information, or may be a separate device, 
such as a remote control device. Recording device 34 6 
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is selected depending on the particular type of 
information desired to be recorded. In a preferred 
embodiment, recording device 346 is a VCR utilized to 
record video and/or audio information transmitted as a 
5 standard television broadcast or a compressed direct 
television broadcast. Recording device 346 may be an 
integral part of the system as shown, or may be a 
separate standard recording device which communicates 
with the control logic via a standard interface. 

10 Thus, it is understood, that while the forms 

of the invention herein shown and described include the 
best mode contemplated for carrying out the invention, 
they are not intended to illustrate all possible forms 
thereof. It will also be understood that the words used 

15 are descriptive rather than limiting, and that various 
changes may be made without departing from the spirit 
and scope of the invention disclosed. 
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What Is Claimed Is : 

1. A system for controlling a recording 
device to effect selective recording of an incoming 
information stream, the system comprising: 

5 a memory for storing at least a portion of the 

incoming information stream; and 

control logic in communication with the memory 
for performing sequential pattern matching of the 
incoming information stream and generating a signal for 
10 the recording device to effect selective recording of 
the incoming information stream based on the sequential 
pattern matching. 

2. The system of claim 1 wherein the control 
logic performs sequential pattern matching based on a 

15 statistical characterization of at least a portion of 
the incoming information stream. 

3. The system of claim 2 wherein the control 
logic performs sequential pattern matching based on a 
block unitary transform of at least a portion of the 

2 0 incoming information stream. 

4 . The system of claim 1 wherein the control 
logic is further operative to learn a sequential 
information pattern used to pattern match subsequent 
information sequences within the incoming information 

25 stream. 

5. The system of claim 4 wherein the control 
logic automatically learns sequential information 
patterns and selects those patterns to be retained based 
on a predetermined number of successful pattern matches 

3 0 within a predetermined time period. 
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6. The system of claim 1 wherein the control 
logic is further operative to learn a sequential 
information pattern used to pattern match subsequent 
information sequences based on a statistical 

5 characterization of the sequential information pattern 
and the subsequent information sequences. 

7. The system of claim 1 wherein the control 
logic comprises a programmed microprocessor. 

8. The system of claim 1 wherein the incoming 
10 information stream represents video information. 

9. The system of claim 1 further comprising 
an input device in communication with the control logic 
for marking incoming information. 

10. The system of claim 1 wherein the control 
15 logic performs sequential pattern matching by computing 

a statistical vector representative of a portion of the 
incoming information stream, matching that first 
statistical vector to a similar stored r statistical 
vector, computing a second statistical vector 
20 representative of the next portion of the incoming 
information stream, and matching that second statistical 
vector to a similar second stored statistical vector, 
wherein the first stored statistical vector identifies 
the second stored statistical vector. 

25 11. A method for controlling a recording 

device to effect selective recording of an incoming 
information stream, the method comprising: 

storing at least a segment of an information 

stream; 
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performing sequential pattern matching of the 
incoming information stream to the stored information 
segment ; and 

generating a signal based on the sequential 
5 pattern matching for the recording device to effect 
selective recording of the incoming information stream. 

12. The method of claim 11 wherein the step 
of storing comprises representing the segment of the 
information stream using summary statistics. 



10 13 . The method of claim 11 wherein the step 

of storing comprises representing the portion of the 
incoming information stream using a block unitary 
transform. 



14 . The method of claim 11 wherein the 
15 information streams include a plurality of information 
frames and wherein the step of storing comprises: 

computing a statistical vector to represent 
each of the plurality of information frames; and 

storing the sequence of statistical vectors in 
20 an indexable table. 



15. The method of claim 14 wherein the step 
of generating a signal comprises generating a control 
signal for recording by the recording device, the 
control signal being operative to accelerate playback of 
25 the recorded information stream. 



16 . The method of claim 14 wherein the step 
of generating a signal comprises generating a control 
signal which causes the recording device to begin 
recording the incoming information stream. 



WO 96/27983 



PCTAJS96/02579 



-46- 

17. The method of claim 11 wherein the 
incoming information stream includes a plurality of 
information frames, the method further comprising 
marking a portion of the incoming information stream 

5 containing at least one of the plurality of information 
frames, prior to the step of storing, for use by the 
step of storing. 

18. The method of claim 17 wherein the step 
of storing comprises automatically extending the marked 

10 portion based on a matching portion of the incoming 
information stream. 

19. The method of claim 11 further comprising 
automatically marking a portion of the incoming 
information stream based on the step of performing 

15 sequential pattern matching. 

20. A system for selective recording of 
received time-based information frames, the system 
having a memory for storing a representation of the 
received time-based information frames within a data 

20 structure, the memory being in communication with 
control logic operative to determine the representation 
of the frames and to generate a signal for controlling 
a recording device based on sequential pattern matching 
of the representation of the received information frames 

25 and previously stored representations of time-based 
information frames, the data structure being 
characterized by: 

a table indexable by both content and ordering 
of the representations of the received time-based 

30 information frames. 
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AMENDED CLAIMS 

[received by the International Bureau on 01 July 1996 (01.07.96): 
original claims 1-20 replaced by amended claims 1-43 (10 pages)] 

1 . A system for controlling a recording 
device to effect selective recording of an incoming 
information stream' having a plurality of information 
5 frames, the system comprising: 

a memory for storing at least a portion of the 
plurality of information frames; and 

control logic in communication with the memory 
for performing sequential pattern matching by comparing 
10 at least two of the plurality of frames to a previously 
stored representation of a sequence of information 
frames and generating a signal for the recording device 
to effect selective recording of the incoming informa- 
tion stream based on the sequential pattern matching. 

15 2. The system of claim 1 wherein the control 

logic performs sequential pattern matching based on a 
plurality of statistical description vectors each repre- 
senting at least a portion of the incoming information 
stream. 

2 0 3. The system of claim 2 wherein the control 

logic performs sequential pattern matching based on a 
block unitary transform of at least a portion of the 
incoming information stream. 

4 . The system of claim 1 wherein the control 
25 logic is further operative to learn a sequential infor- 
mation pattern used to pattern match subsequent informa- 
tion sequences within the incoming information stream 
wherein the sequential information pattern comprises a 
plurality of statistical description vectors each 
30 representing one of the plurality of information frames. 



AMENDED SHEET (ARTICLE 



WO 96/27983 



PCT/US96/02579 



48 



5. The system of claim 1 wherein the control 
logic automatically learns sequential information 
patterns and selects at least some of the sequential 
information patterns to be retained based on a predeter- 

5 mined number of successful pattern matches within a 
predetermined time period. 

6 . The system of claim 1 wherein the control 
logic is further operative to learn a sequential infor- 
mation pattern used to pattern match subsequent informa- 

10 tion sequences based on a statistical characterization 
of the sequential information pattern and the subsequent 
information sequences. 

7. The system of claim l wherein the control 
logic comprises a programmed microprocessor. 

15 8- The system of claim 1 wherein the control 

information stream represents video information. 

9. The system of claim 1 further comprising 
an input device in communication with the control logic 
for marking incoming information. 

20 10. The system of claim 1 wherein the control 

logic performs sequential pattern matching by computing 
a first statistical vector representative of a first one 
of the plurality of information frames, matching the 
first statistical vector to a similar stored statistical 

25 vector, computing a second statistical vector represen- 
tative of the next one of the plurality of information 
frames, and matching the second statistical vector to a 
similar second stored statistical vector, wherein the 
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first stored statistical vector identifies the second 
stored statistical vector. 

11 . A method for controlling a recording 
device to effect selective recording of an incoming 
5 information stream having a plurality of information 
frames, the method comprising: 

storing the plurality of information frames; 

for each of the plurality of information 
frames, computing a corresponding statistical descrip- 
10 tion vector; 

performing sequential pattern matching of the 
incoming information stream to the stored information 
segment wherein sequential pattern matching includes 
comparing at least two of the plurality of information 
15 frames based on the corresponding statistical descrip- 
tion vectors; and 

generating a signal based on the sequential 
pattern matching for the recording device to effect 
selective recording of the incoming information stream. 

20 12. The method of claim 11 wherein the step 

of storing comprises representing the segment of the 
information stream using summary statistics. 

13 . The method of claim 11 wherein the step 
of computing a corresponding statistical description 

25 vector comprises representing each of the plurality of 
information frames of the incoming information stream 
using a block unitary transform. 

14 . The method of claim 11 wherein the step 
of storing comprises: 
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storing the statistical description vectors in 
an indexable table. 

15. The method of claim 14 wherein the step 
of generating a signal comprises generating a control 

5 signal for recording by the recording device, the 
control signal being operative to accelerate playback of 
the recorded information stream. 

16. The method of claim 14 wherein the step 
of generating a signal comprises generating a control 

10 signal which causes the recording device to begin 
recording the incoming information stream. 

17. The method of claim 11 further comprising 
marking a portion of the incoming information stream 
containing at least one of the plurality of information 

15 frames, prior to the step of storing, for use by the 
step of storing. 

18. The method of claim 17 wherein the step 
of storing comprises automatically extending the marked 
portion based on a matching portion of the incoming 

20 information stream. 

19. The method of claim 11 further comprising 
automatically marking a portion of the incoming informa- 
tion stream based on the step of performing sequential 
pattern matching. 

25 20. A system for selective recording of a 

plurality of received sequential time-based information 
frames, the system having a memory for storing a repre- 
sentation of each of the received time-based information 
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frames within a data structure, the memory being in 
communication with control logic operative to determine 
the representation of each of the frames and to generate 
a signal for controlling a recording device based on 
5 sequential pattern matching of the representations of 
the received information frames and previously stored 
representations of a sequence of time-based information 
frames, the data structure comprising: 

a table indexable by both content and ordering 
10 of the representations of the received time-based 
information frames. 

21. The system of claim 2 wherein the plural- 
ity of statistical description vectors represents a 
sequence of the plurality of information frames. 

15 22 - The method of claim 11 wherein the 

plurality of statistical description vectors represents 
a sequence of the plurality of information frames. 

23. The system of claim 1 wherein the control 
logic generates a control signal for the recording 
device to omit recording of data from the incoming 
information stream. 



20 



24. The system of claim 23 wherein the 
incoming information stream contains an advertisement. 

25. The system of claim 1 wherein the control 
25 logic generates a control signal for the recording 

device to record a segment of the incoming information 
stream. 
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26. The system of claim 25 wherein the 
incoming information stream represents one element of a 
related series of elements. 

27. The method of claim 11 wherein the step 
of generating a signal comprises generating a signal to 
stop recording of the incoming information stream. 

28. The method of claim 27 wherein the 
incoming information stream is a commercial advertise- 
ment. 

29. a system for controlling a recording 
device to omit selected information within an incoming 
information stream having a plurality of information 
frames, the system comprising: 

a memory for storing the plurality of incoming 
15 information frames stream; and 

control logic in communication with the memory 
for performing sequential pattern matching by comparing 
at least two of the plurality of frames to a previously 
stored sequence of information frames and generating a 
signal for the recording device to omit selected infor- 
mation from being recorded by the recording device. 

30. The system of claim 29 wherein the 
control logic performs sequential pattern matching 
comprising generating a corresponding statistical 
description vector for each of the plurality of informa- 

tion frames. 

31. The system of claim 30 wherein at least 
two of the plurality of statistical description vectors 
represents a sequence of information frames. 



20 



25 
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32. The system of claim 30 wherein each 
statistical description vector includes a plurality of 
statistical elements and wherein the control logic 
terminates comparison of the at least two frames based 
on a difference between any corresponding ones of the 
plurality of statistical elements to improve efficiency 
of comparisons. 

33. A method for recording a segment of data 
within an incoming information stream having a plurality 
of sequential information frames, the method comprising: 
computing a plurality of statistical descrip- 
tion vectors each corresponding to one of the plurality 
of information frames; 

storing the plurality of statistical descrip- 
tion vectors including an indication of order among the 
sequential information frames in a table indexable by 
content and order; 

comparing at least two of the plurality of 
information frames based on the corresponding statisti- 
cal description vectors; and 

generating a signal to record the segment of 
data based on the step of comparing. 

34. The method of claim 33 wherein the at 
least two information frames include a first and a 
second information frame and wherein the step of compar- 
ing comprises comparing the second of the at least two 
frames based on information generated by comparing the 
first of the at least two frames. 

35. The method of claim 34 wherein the at 
least two information frames includes N information 
frames, N representing a positive integer greater than 
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rsr each of the ~ : ~ ™ir 

a mdexmg a current frame and having a value w 
unity and N, inclusive. between 

36. a method for recording = « 

inform- P iu »lity of information channels, the 

information stream having a pluralitv n * 
information frames the JL a P Xu «Hty of sequential 
meB ' tne method comprising: 
selecting one of the plural it- v ■ * 
channels; Plurality of information 

computing a plurality of statistical descrir, 
txco vectors each corresponding to one of the plura^y 
of information frames; Plurality 

rrames to a corresponding at least fun 
Piously stored sequential information frame . 
the corresponding statistical description vectors- 

storing ve r c e to e r a s tin a 9 nd th l ^ " Sele " in9 ' 
number of T comparing until a predetermined 

number of sequential information frames match a corre- 
sponding number of the previously stored sequel 
information frames; and sequential 

generating a signal to record the segment of 



data. 



37. The method of claim 36 wherein the 

HZ! of data is an install " eM - * -*«~ 
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38. The method of claim 36 wherein the 
segment of data is one of a series of related programs. 

39. The method of claim 36 wherein the at 
least two of the plurality of information frames in- 
cludes a first and a second frame and wherein the step 
of comparing comprises searching only a subset of the 
previously stored sequential information frames based on 
a successful match of the first information frame. 

40. The method of claim 39 wherein the step 
of comparing comprises searching a subset of the previ- 
ously stored statistical description vectors wherein the 
subset is reduced in number by each previous successful 
match. 

41. The method of claim 36 wherein the step 
of selecting comprises randomly selecting one of the 
plurality of information channels. 



42. The method of claim 36 wherein the step 
of selecting comprises selecting based on at least one 
previously successful match of at least two information 
20 frames. 



43. A system for controlling a recording 
device to effect selective recording of an incoming 



information stream, the system comprising: 

a memory for storing at least a portion of the 
incoming information stream; and 

control logic in communication with the memory 
for performing sequential pattern matching by computing 
a first statistical vector representative of a fi rst 
portion of the incoming information stream, matching the 
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first statistical vector to a similar stored statistical 
vector, computing a second statistical vector represen- 
tative of the next portion of the incoming information 
stream, matching the second statistical vector to a 
5 similar second stored statistical vector, the first 
stored statistical vector identifying the second stored 
statistical vector, and generating a signal for the 
recording device to effect selective recording of the 
incoming information stream based on the sequential 
10 pattern matching. 
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STATEMENT UNDER ARTICLE 19 



Applicant hereby amends the claims of the Interna- 
tional Application to coincide with the claims in the United 
States patent application, claims 1-43 are the same as those 
presently on file before the U.S. Patent Office in the 
corresponding priority application. 
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